NONLINEAR SIGNAL PROCESSING BASED ON REPRODUCING KERNEL HILBERT SPACE By JIANWU XU A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy NONLINEAR SIGNAL PROCESSING BASED ON REPRODUCING KERNEL HILBERT SPACE By Jianwu Xu December 2007 Chair: Jose C. Principe Major: Electrical and Computer Engineering My research aimed at analyzing the recently proposed correntropy function and presents a new centered correntropy function from time-domain and frequency-domain approaches. It demonstrats that correntropy and centered correntropy functions not only capture the time and space structures of signals, but also partially characterize the higher order statistical information and nonlinearity intrinsic to random processes. Correntropy and centered correntropy functions have rich geometrical structures. Correntropy is positive definite and centered correntropy is non-negative definite, hence by Moore-Aronszajn theorem they uniquely induce reproducing kernel Hilbert spaces. Correntropy and centered correntropy functions combine the data dependent expectation operator and data independent kernels to form another data dependent operator. Correntropy and centered correntropy functions can be formulated as “generalized” correlation and covariance functions on nonlinearly transformed random signals via the data independent kernel functions. Those nonlinearly transformed signals appear on the sphere in the reproducing kernel Hilbert space induced by the kernel functions if isotropic kernel functions are used. The other approach is to directly work with the reproducing kernel Hilbert space induced by the correntropy and centered correntropy functions directly. The nonlinearly transformed signals in the reproducing kernel Hilbert space is no longer stochastic but rather deterministic. The reproducing kernel Hilbert space induced by the correntropy and centered correntropy functions includes the expectation operator as embedded vectors. The two views further our understandings of correntropy and centered correntropy functions in geometrical perspective. The two reproducing kernel Hilbert space induced by kernel functions and correntropy functions respectively represent stochastic and deterministic functional analysis. The correntropy dependence measure is proposed based on the correntropy coefficient as a novel statistical dependence measure. The new measure satisfies all the fundamental desirable properties postulated by Renyi. We apply the correntropy concept in pitch determination, and nonlinear component analysis. The correntropy coefficient is also employed as a novel similarity measure to quantify the inder-dependencies of multi-channel signals. CHAPTER